feat(speechify): streaming TTS with word-level timestamps via official SDK#6327
Open
luke-speechify wants to merge 9 commits into
Open
feat(speechify): streaming TTS with word-level timestamps via official SDK#6327luke-speechify wants to merge 9 commits into
luke-speechify wants to merge 9 commits into
Conversation
|
|
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
luke-speechify
marked this pull request as draft
July 6, 2026 12:56
…l SDK Upgrade the Speechify plugin to stream audio with aligned word-level timestamps. Synthesis now uses the official speechify-api SDK and chunks streamed input into sequential /audio/speech calls, emitting audio and timed transcripts per sentence (streaming + aligned_transcript capabilities). Route maintenance to the Speechify-owned repository.
luke-speechify
force-pushed
the
lo/speechify-streaming-tts
branch
from
July 6, 2026 13:23
1745975 to
c008b99
Compare
luke-speechify
marked this pull request as ready for review
July 13, 2026 20:29
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
luke-speechify
force-pushed
the
lo/speechify-streaming-tts
branch
from
July 13, 2026 21:15
d84105d to
dff6ed7
Compare
luke-speechify
added a commit
to luke-speechify/agents
that referenced
this pull request
Jul 13, 2026
Addresses Devin review feedback on livekit#6327: - SynthesizeStream: flush after each sentence's audio push. Without this, the AudioEmitter's tail buffer holds back the last ~10ms of each sentence until the next arrives, adding gaps between sentences and undermining the near-streaming promise. Matches the canonical stream_adapter pattern. - _raise_from: only wrap ConnectionError as APIConnectionError (which the framework retries). Previously any Exception - base64 decode failures, AttributeError from an unexpected response shape - was misclassified as a retryable connection error, so the framework would retry a request that would deterministically fail again. Unknown exceptions now propagate as-is.
luke-speechify
added a commit
to Speechify-AI/demos
that referenced
this pull request
Jul 13, 2026
speechify-api v3.0.0 renamed the constructor kwarg from api_key to token. - audiobook-pipeline/requirements.txt: speechify-api>=2.0.0 -> >=3.0.1 - audiobook-pipeline/main.py: api_key -> token (env lookup + guard + Speechify(token=...)) - pipecat-agent-speechify/verify_stream.py: api_key -> token in the raw-HTTP smoke test (variable + Bearer header f-string) for symmetry with the SDK side; this script does not use the SDK but its local variable was still named api_key - livekit-agent-speechify-python/README.md: the note on the plugin's public API says `token=` now (matches livekit/agents#6327's renamed param)
Editorial year was regressed 2024 -> 2023 while making unrelated content changes. Restoring the year main already carries.
…GetVoice re-export) The SDK v3.0.0 renamed the auth kwarg from api_key to token and the type-check job was failing on the v2-era `AsyncSpeechify(api_key=...)` call plus the implicit `GetVoice` re-export from tts.py. Bumps the pin so old resolutions don't sneak the v2 shape back in. - pyproject.toml: speechify-api>=2.0.0 -> >=3.0.1 - tts.py: rename public `api_key` param to `token`; pass through to AsyncSpeechify(token=...); update docstring + error message - __init__.py: re-export GetVoice from speechify.types.get_voice directly instead of laundering through .tts (fixes mypy no-implicit-reexport) - uv.lock: refreshed with speechify-api 3.0.1
Addresses Devin review feedback on livekit#6327: - SynthesizeStream: flush after each sentence's audio push. Without this, the AudioEmitter's tail buffer holds back the last ~10ms of each sentence until the next arrives, adding gaps between sentences and undermining the near-streaming promise. Matches the canonical stream_adapter pattern. - _raise_from: only wrap ConnectionError as APIConnectionError (which the framework retries). Previously any Exception - base64 decode failures, AttributeError from an unexpected response shape - was misclassified as a retryable connection error, so the framework would retry a request that would deterministically fail again. Unknown exceptions now propagate as-is.
luke-speechify
force-pushed
the
lo/speechify-streaming-tts
branch
from
July 13, 2026 21:46
eeffbd5 to
fb81a91
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Upgrades the
livekit-plugins-speechifyTTS plugin to stream audio with word-level timestamps, using Speechify's officialspeechify-apiSDK.Previously the plugin was non-streaming (
streaming=False) and used rawaiohttpagainst/audio/stream(binary audio only, no timing metadata). Speechify's word-level speech marks are only returned by the batch/audio/speechendpoint, so this implementation chunks streamed input into per-sentence sequential/audio/speechcalls, emitting audio and aligned word timestamps as each sentence completes — giving near-streaming time-to-first-audio andaligned_transcriptsupport.Changes
TTSCapabilities(streaming=True, aligned_transcript=True)SynthesizeStream: sentence-chunked sequential synthesis with cumulative word-mark time offsetsChunkedStream: one-shot synthesis with word marksspeechify-apivendor SDK instead of hand-rolled HTTPlivekit-plugins-gnaniprecedent):Repository/Issuespoint at Speechify's maintenance repoMaintenance & distribution
This plugin is maintained by Speechify and distributed by LiveKit as part of
livekit/agents. Speechify keeps a mirror of the plugin source at Speechify-AI/livekit-plugin-python for maintenance and issue triage, and proposes changes upstream here. The canonical, user-installable package remainslivekit-plugins-speechify, published by LiveKit — there is no separate Speechify-published package.Verification
ruff check+ruff format --checkcleanmypycleantests/test_tts.pyround-tripHappy to adjust to match maintainer preferences.